Although applicable to any system that moves on a surface with kerbs, like vehicles, robots, motion assistance systems for visually impaired people or wheelchairs, the present invention will mainly be described in combination with vehicles.
In modern vehicles a plurality of driver assistance systems provide different assistance functions to the driver. Such driver assistance function range from automatic wiper activation functions to semi-automatic or automatic driving functions for a vehicle.
For complex driver assistance functions, especially where semi-automatic or automatic driving of the vehicle is involved, the respective driver assistance system needs to gather information about the vehicles surroundings, e.g. to avoid collisions when automatically parking a vehicle.
In such driver assistance systems a wide range of sensors can be employed to build a model of the vehicles surroundings. E.g. ultra-sonic distance sensors can be used to measure the distance between the vehicle and an obstacle, or cameras can be used to detect the positions and types of obstacles.
Semi-automatic or automatic driving furthermore requires detection of road limits or kerbs on the road, wherein a kerb represents any physical limit of the road. Knowing such limits and differentiating such limits from simple lines painted on the road is essential in navigation functions of the car control systems, such as lane keeping, autonomous parking, prevention of damage of wheels and vehicle body and the like.
The main types of sensors that are being used for kerb detection in vehicles are: mechanical sensors, infrared sensors, ultrasonic sensors, laser scanners and 3D-cameras.
Each type of sensor has its applicability limits, e.g. regarding resolution, accuracy, response time, power consumption, and cost.
The 3D-camera based solutions usually detect kerbs fairly accurately, but the required computer algorithms use intensive image processing and thus have high computational costs.
A 3D-camera based kerb detection system is e.g. disclosed in document DE 10 2011 056 671 A1, which is incorporated by reference.
Typical 3D-camera based kerb detection systems usually comprise:
1) Kerb hypotheses generation. This step is performed in a single image, wherein nearby parallel line segments (edges) identified by standard edge detectors are clustered to form an initial kerb hypotheses.
2) Validation in 3D. This step is accomplished by using a 3D-camera or stereo-camera. A feature matching algorithm analyses images of the left camera and the right camera in the stereo-rig and generates a set of 3D-points. Considering 3D-point sets on both sides of the kerb hypothesis line, one can determine if there is a real 3D-step between these point sets. If a 3D-step exists, the candidate line is classified as kerb. Otherwise, the candidate line belongs to any planar object in the scene, e.g. it can be a part of road markings painted on the road surface.3) Validation in time. This step is applied to: a) increase confidence for each detected kerb line, and b) to connect shorter kerb segments into the longer kerb models, and c) to establish tracking capabilities in the system, so that the kerb candidates in the next frame are searched in a restricted area predicted from the previous frames.
The main element of this processing pipeline is a stereo-matching algorithm producing a 3D-point cloud from which the main feature of the kerb, a 3D-step, can be extracted by a robust model fitting algorithm. Such algorithms can only be used with calibrated stereo-rigs, which are not always available in vehicles, and require high computing power.
Accordingly, there is a need for a method and an apparatus for providing a kerb detection system which can be used with simple cameras and requires less computational power.